Search Results: "anderson"

8 April 2014

Dirk Eddelbuettel: RcppArmadillo 0.4.200.0

Conrad released a new upstream release 4.200 for Armadillo, his templated C++ library for linear algebra, earlier today. As usual, this was rolled up in a new RcppArmadillo release 0.4.200.0; I had actually made two pre-releases leading up his 4.200 release and was hence not expecting any surprises. Prior to uploading to CRAN, I also tested against all 61 reverse dependencies of the package. Last but not least, I also updated the corresponding Debian package.
Changes in RcppArmadillo version 0.4.200.0 (2014-04-07)
  • Upgraded to Armadillo release Version 4.200 (Flintlock Swoop)
    • faster transpose of sparse matrices
    • more efficient handling of aliasing during matrix multiplication
    • faster inverse of matrices marked as diagonal
Courtesy of CRANberries, there is also a diffstat report for the most recent release As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

8 March 2014

Dirk Eddelbuettel: RcppArmadillo 0.4.100.2.1

Conrad made a first bug-fix release 4.100.2 for Armadillo, his templated C++ library for linear algebra, yesterday. See below for details on the the small set of changes. This was quickly wrapped up in a new RcppArmadillo release 0.4.100.2.1 release. I also added some first steps towards turning C++11 on. This can be used now with a pre-release of R 3.1.0, and can be generally activated once R 3.1.0 is released on April 10.
Changes in RcppArmadillo version 0.4.100.2 (2014-03-07)
  • Upgraded to Armadillo release Version 4.100.2
    • fix for handling null vectors by normalise()
    • fix for memory handling by sparse matrices
  • Correct use of [[ depends()]] in skeleton example file
  • Prepare src/Makevars for C++11 support from R 3.1.0 by defining USE_CXX11 which is currently commented out
  • In the Armadillo configuration, turn on C++11 support if USE_CXX11 is defined
Courtesy of CRANberries, there is also a diffstat report for the most recent release As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

1 March 2014

Dirk Eddelbuettel: RcppArmadillo 0.4.100.0

A new minor upstream release 4.100 of the Armadillo templated C++ library for linear algebra was finalised by Conrad yesterday. I rolled up the corresponding RcppArmadillo release 0.4.100.0 which is now on CRAN and will go to Debian shortly.
Changes in RcppArmadillo version 0.4.100.0 (2014-02-28)
  • Upgraded to Armadillo release Version 4.100.0 (Dirt Cruiser)
    • added normalise() for normalising vectors to unit p-norm
    • extended the field class to handle 3D layout
    • extended eigs_sym() and eigs_gen() to obtain eigenvalues of various forms (eg. largest or smallest magnitude)
    • automatic SIMD vectorisation of elementary expressions (eg. matrix addition) when using Clang 3.4+ with -O3 optimisation
    • faster handling of sparse submatrix views
    • workaround for a bug in LAPACK 3.4
Courtesy of CRANberries, there is also a diffstat report for the most recent release As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

28 February 2014

Russell Coker: Links February 2014

The Economist has an interesting and informative article about the lack of reproducability of scientific papers and the implications for scientific research [1]. Regina Dugan gave an interesting TED talk about some of the amazing DARPA projects [2]. Chris Anderson interviewed Elon Musk about the Tesla cars, SpaceX, and his new venture Solar City [3]. Elon has a lot of great ideas for improving humanity while also making money. Smart Planet has an interesting article about Bhutan s switch to electric vehicles [4]. Paul Piff gave an insightful and well researched TED talk about the ways that money makes people mean [5]. Maryn McKenna wrote an interesting article for Wired about what happens when the current anti-biotics stop working [6]. Unfortunately she lists increasing food prices as a consequence, really the unreasonably low price of meat is due to the misuse of anti-biotics that is causing this problem. Linda Walther Tirado wrote an interesting article about being poor titled Why I Make Terrible Decisions, or, Poverty Thoughts [7]. It gives a real insight into the situation of people who are trapped in poverty. When someone who is as obviously intelligent as Linda feels that it s impossible to escape poverty there is a real problem in the system. While Australia doesn t suck nearly as badly as the US in this regard (higher minimum wage and better health care) we still need to improve things, I know people in Australia who s experience bears some similarity to Linda s. Maxwell Neely-Cohen wrote an interesting article about peer pressure [8]. Some of the conclusions are dubious, but the ideas on the way the Internet changes peer relationships in high school are interesting. An English pediatrician wrote an article for The Daily Beast about why he won t accept anti-vac clients [9]. There are some decent people in the Liberal Party, Liberal MP Warren Entsch attacks Cory Bernardi on gay obsession [10]. AFAIK we haven t yet had a gay sex scandal involving a homophobic Australian politician

7 January 2014

Dirk Eddelbuettel: RcppArmadillo 0.400

A new major release 4.000 of Armadillo came out out yesterday morning. This is a new stable release by Conrad with a new major number, and it brings a few nice changes as detailed below. I rolled up RcppArmadillo version 0.400 yesterday morning, and following a little bit of discussion this version is now on CRAN and in Debian. The complete list of changes is below.
Changes in RcppArmadillo version 0.4.000 (2014-01-05)
  • Upgraded to Armadillo release Version 4.000 (Feral Steamroller)
    • added eigen decompositions of sparse matrices: eigs_sym() and eigs_gen() [ but this requires linking against ARPACK which RcppArmadillo as a pure-template package does not do, and R is not linked against ARPACK either. ]
    • added eigen decomposition for pair of matrices: eig_pair()
    • added simpler forms of eig_gen()
    • added condition number of matrices: cond()
    • expanded find() to handle cubes
    • expanded subcube views to access elements specified in a vector
    • template argument for running_stat_vec expanded to accept vector types
    • more robust fast inverse of 4x4 matrices
    • faster divide-and-conquer decompositions are now used by default for eig_sym(), pinv(), princomp(), rank(), svd(), svd_econ()
    • the form inv(sympd(X)) no longer assumes that X is positive definite; use inv_sympd() instead
    • added MEX connector for interfacing Octave/Matlab with Armadillo matrices (contributed by George Yammine)
Courtesy of CRANberries, there is also a diffstat report for the most recent release As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

10 December 2013

Dirk Eddelbuettel: RcppArmadillo 0.3.930.1

A new Armadillo release 3.930 came out a few days ago, with a very nice set of changes (see below). I rolled this into RcppArmadillo 0.3.930.0. However, one of these changes revealed that R shipped only the standard SVD for complex-valued matrices, and not the more performant divide-and-conquer approach. So in R builds using the default built-in Lapack, at least one CRAN package no longer built. After some back and forth, Conrad put some branching in the library to fall back to the standard SVD, and I added a built-time configuration test for an appropriate preprocessor directive used by the fallback code. This is now on which is now on CRAN and in Debian as RcppArmadillorelease 0.3.930.1, and Conrad will probably update the Armadillo page as well (though the fix is only needed with R's builtin Rlapack). Also of note is that R Core already added the missing Fortran routine zgesdd to R 3.1.0 (aka "R-devel") so this issue goes away with the next release. Also of note, I wrote up a short Rcpp Gallery post illustrating the performance gains available from divide-and-conquer SVD. The complete list of changes is below.
Changes in RcppArmadillo version 0.3.930.1 (2013-12-09)
  • Upgraded to Armadillo release Version 3.930.1
    • Armadillo falls back to standard complex svd if the more performant divide-and-conquer variant is unavailable
  • Added detection for Lapack library and distinguish between R's own version (withhout zgesdd) and system Lapack; a preprocessor define is set accordingly
Changes in RcppArmadillo version 0.3.930.0 (2013-12-06)
  • Upgraded to Armadillo release Version 3.930 ("Dragon's Back")
    • added divide-and-conquer variant of svd_econ(), for faster SVD
    • added divide-and-conquer variant of pinv(), for faster pseudo-inverse
    • added element-wise variants of min() and max()
    • added size() based specifications of submatrix view sizes
    • added randi() for generating matrices with random integer values
    • added more intuitive specification of sort direction in sort() and sort_index()
    • added more intuitive specification of method in det(), .i(), inv() and solve()
    • added more precise timer for the wall_clock class when using C++11
  • New unit tests for complex matrices and vectors
Courtesy of CRANberries, there is also a diffstat report for the most recent release As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

20 November 2013

Dirk Eddelbuettel: RcppArmadillo 0.3.920.3

Conrad released a bug fix version 3.920.3 for his Armadillo library. So I prepared a new release 0.3.920.3 of RcppArmadillo which is nor on on CRAN and also already in Debian. Note that we skipped interim version .2 which addressed an Atlas linking issue which doesn't affect us in the R context as we get LAPACK and BLAS via the R process we get used from. The complete list of changes is below.
Changes in RcppArmadillo version 0.3.920.3 (2013-11-20)
  • Upgraded to Armadillo release Version 3.920.3
    • fix for handling of tiny matrices by .swap()
Courtesy of CRANberries, there is also a diffstat report for the most recent release As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

16 September 2013

Debian Med

DebConf 13 report (by Andreas Tille) General impression unofficial  Scenic Hacklab I'm beginning my DebConf report in an unofficial "Scenic Hacklab" right at the edge of the lake in Yverdon. This is the right place to memorise the last days. When I started from this place cycling to Le Camp 12 days ago I was full of great expectations and what should I say - the reality has even beaten these. Once it comes about comparing DebConfs even if it is an unfair comparison due all the differences my secret long term favourite was Helsinki very closely followed by Argentina and also very closely followed by all the other great DebConfs I joined (and I joined all in Europe). Would Le Camp be able to beat it? The short answer is: Yes, it is now my favourite DebConf while I think I do not suffer from the last-Debconf-was-the-best-DebConf-syndrome (and I realised there are others thinking the same). As you might probably know I'm a bit addicted to swimming. While Helsinki had admittedly the better conditions I was at least able to fix the distance issue using my bicycle. (Hey, those Le Camp photographers did a great job in hiding the fact that you can not actually touch the lake right from the meadow of Le Camp.) Being able to have my bicycle at DebConf scored some extra points. However, the really great view of the lake, the inspiring "Scenic Hacklab" which was my favourite place has bumped DebConf13 at first place in my personal ranking. So it comes quite natural to say: "Kudos to the great organisation team!" They did a Swiss-like precise work and perfectly succeeded in hiding any problems (I assume there were some as always) from the attendees so everything went smooth, nice and shiny for the attendees. The local team was even precise in setting up great weather conditions for DebConf. sunrise over  the lake While saying thanks to the local team I would like to also explicitly thank Luca Capello who has quite some share that this DebConf was possible at all (while I have to decrease my DebConf score one point because he was not really there - Luca to bad that you were not able to come full time!) Also thanks to Gunnar and Gannef who helped remotely (another score down because I were missing them this year as well). Even if it was my favourite DebConf I was not able to work down my todo list fully (which was not only uploading one package per day which I at least statistically fullfilled). But that's probably a general feature of todo lists anyway. One item was definitely done: Doing my daily swimming BoF. I actually was able to do the other parts of the triathlon which was skipped by Christian and have done in summary about 150km cycling with 3500m elevation and estimated 7-8km swimming (0m elevation ;-)). Considering the great view at sunrise over the lake I was not hating my "Senile bed escape" disease too much (I was every day waking up at sunset) - it was simply a great experience. I will never forget seeing water drips glimmering like gold inside the morning sun while seeing the Alps panorama in the distant. I hope I was able to help all interested swimmers with the DebConf Beach Map which was just a by-product of my activities in DebCamp. Speaking about OSM: I was astonished that the area was way less covered than I expected. Thanks to several DebConf attendees the situation became better and the map does not only show random trees in the wild but also the tracks leading to these. (Remark: It was no DebConf attendee who is responsible for plastering the map with single trees.) While I had my mapping focus basically close to the edge of the lake I was also able to even map my very own street. :-) I clearly remember one specific mapping tour when I was invited by the DPL: He convinced me to join him on a bicycle tour and since I was afraid to get fired I joined him instead to keep on hacking. Also Sorina was brave enough to join us on the tour and she did quite well. (Sorina, do you remember the agreement about your work on the installer? ;-)) Lucas described the tour as: going uphill on only asphalted roads. Sorina and me were witnessing the mighty DPL powers when we left the wood around Le Camp to reach the described road: The asphalt was just put onto the road - no doubt that it was done on the immediate demand of mighty DPL. :-) DebCamp time was flying like nose dive and a lot of known (and unknown) faces arrived at Le Camp. What I really liked a lot this year was that several really young children has pulled down the average age of DebConf attendees. I clearly remember all the discussion one year ago what to do about children. As always the issue was solved in a typical Debian way: Just do it and bring your children - they had obviously a great time as well. I think the youngest child was 2 months and the oldest "child" above 20. ;-) Actually Baptiste Perrier did great in making the C&W party a success and had obviously a nice time. (I wished my son would have been able to come as well but he needs to write his bachelor s thesis in physics. :-() It was nice to see the kids using all playing facilities and communicating with geeks. Also I would like to point out that even the very young attendees had their share at the success of DebConf: Just think of the three "bell ringing assistants" who helped me ringing the bells for lunch and dinner. I've got this cool job from Didier in the beginning of DebCamp. I must say having some real bells ringing is by far nicer than just the "lunch / dinner starts in 10 minutes" from IRC bot. The only thing I did not understand was that people did not considered ringing the bells at 8:00 for breakfast as a good idea. Regarding the food in general I would also like to send kudos to the kitchen: It was tasty, freshly prepared, regional food with a good change rate. I really liked this. Extra points for having the chance to sit outside when eating. Talks But lets have a look into the conference programme. I'd really recommend watching the videos of the talks Bits from the DPL (video) and Debian Cosmology (video). I considered both talks as entertaining and interesting. I also really hope that the effort Enrico Zini started in Debian Contributors (video) will be successful. I had some talks and BoFs myself starting with Why running a Blend (video) and I admit that (as usual) the number of attendees was quite low even if I think there is some proof (see below) that it is interesting for way more people who should consider working more "blendish" in their team. Do you know how to recruit one developer per year and relax the man power problem in your team? Feel free to watch the video. We have confirmation that ten DDs of our team have considered to join Debian only because Debian Med exists. Admittedly biology and medicine are really leaf topics inside the Debian universe. So if even this topic that has a very tiny share of the Debian users is able to attract this level of attention - how many more people could we win for multimedia, games, GIS and others? So if you feel you are quite overworked with your packaging and you have no time this is most probably wrong. The amount of time is basically a matter of priorities you set for your tasks. Try to put some higher priority onto using the just existing Blends tools I explained in my talk to attract more users and developers to your team and by doing so spread the workload over more people. It works, the prove was given in my main talk. So before you start working on a specific package you should wonder who else could have an even stronger interest to get this work done and provide him with some additional motivation and help to get the common goal done. The interesting thing is that my BoF about How to attract new developers for your team (video) - which was a simple report about some by-product of the Blends work - made it into the main talk room and got way more attention. For me this is the proof that the Blends concept itself is probably badly perceived as something like "a few outsiders are doing damn specific stuff which is not really interesting for anybody else" instead of what is really is: Smoothing the way from specific upstream applications to the end user via Debian. Once you see the video of this BoF you can observe how my friend Asheesh Laroia became more and more excited about the Blends concept and admitted what I said above: We should have more Blends for different fields. Funnily enough Asheesh asked me in his excitement to talk more about Blends. This would have been a really good suggestion ten years ago. At DebConf 3 in Oslo I had my very first talk about Blends (at this time under the name "Debian Internal Projects"). I continuously kept on talking about this (MiniDebConf Peking 2005, DebConf 5, Helsinki (video), DebConf 7, Edinburgh (video), DebConf 8, Mar del Plata (video), DebConf 9, C ceres (video), MiniDebConf Berlin 2010 (video in German), MiniDebConf Paris 2010 (not video recorded), DebConf 11, Banja Luka (video) ... and these are only (Mini)DebConfs my talks page is full of this topic) and every new year I try different ways to communicate the idea to my fellow Debianistas. I'm wondering how I could invent a title + abstract avoiding the term Blends, put "Git", "release" and "systemd versus upstart" in and being able to inform about Blends reasonably by not becoming to off topic with the abstract. I also registered the Debian Science round table. I admit we were lacking some input from remote via IRC which used to be quite helpful in the past. The attendees agreed upon the handling of citations in debian/upstream files which was invented by Debian Med team to create even stronger bounds to our upstream developers by giving their work extra reward and providing users with even better documentation (see my summary in Wiki). As usual I suggested to create some Debian Science offsprings like "Debian Astronomy", "Debian Electronics", "Debian Mathematics", "Debian Physics" etc. who could perfectly leave the Debian Science umbrella to get a more fine grained structure and a more focused team to enhance the contact to our users. Unfortunately there is nobody who volunteers to take over the lead for such Blends. I have given a short summary about this BoF on the Debian Science mailing list. In the Debian Med meeting I have given some status report. No other long term team members were attending DebConf and so I gave some kind of introduction for newcomers and interested people. I touched also the DebiChem topic which maintains some packages that are used by biologists frequently and so we have a good connection to this team. Finally I had registered three BoFs in Blends I'm actually not (or not yet) active part of. My motivation was to turn the ideas I have explained in my main talk into specific application inside these teams and helping them to implement the Blends framework. In the first BoF about Debian GIS I have shown the usual team metrics graphs to demonstrate, that the one packaging team Pkg-OSM is in danger to become MIA. There are only three persons doing actual uploads. Two of them were at DebConf but did not joined the BoF because they do not consider their contribution to Pkg-OSM as a major part of their general Debian work. I will contact the main contributor David Paleino about his opinion to move the packages step by step into maintenance of Debian GIS packaging team to try to overcome the split of two teams that are sharing a good amount of interest. At least if I might become an Uploader for one of the packages currently maintained by Pkg-OSM I will move this to pkg-grass-devel (which is the name of the packaging team of Debian GIS for historical reasons). The attendees of the BoF have considered this plan as sensible. Moreover I talked about my experiences with OSGeo Live - an Ubuntu derivative that tries to provide a full tool chain to work on GIS and OSM problems ... basically the same goal as Debian GIS has just provided by the OSGeo project. I'm lurking on OSGeo mailing list when I asked explicitly I've got the answer that they are working together with Debian GIS and are using common repository (which is IMHO the optimal way of cooperation). However, it seems that several protagonists of OSGeo Live are underestimating the resources provided by Debian. For instance there was a question about Java packaging issues but people were not aware about the existence of the debian-java mailing list. I was able to give an example how the Debian Med team managed to strengthen its ties to BioLinux that is also an Ubuntu derivative for biologists. At our first Debian Med sprint in 2011 we invited developers from BioLinux and reached a state where they are using the very same VCS on Alioth where we are maintaining our packages. At DebConf I was able to upload two packages where BioLinux developers did certain changes for enhancing the user experience. My "work" was just bumping the version number in changelog and so we did profit from the work of the BioLinux developers as well as they are profiting from our work. I plan to dive a bit more into Debian GIS and try to strengthen the connection to OSGeo Live a bit. The next BoF was the Debian Multimedia meeting. It was nice that the current leader of Ubuntu Studio Kaj Ailomaa joined the meeting. When I was explaining my ideas about cooperation with derivatives I repeated my detailed explanation about the relation with BioLinux. It seems every topic you could cover inside Debian has its related derivative. So to me it seems to be quite natural to work together with the developers of the derivative to join forces. I actually consider a Blend a derivative done the right way = inside Debian. The final work for the derivers that might be left for them is doing some shiny customising of backgrounds or something like this - but all the hard work could and should be done in common with the relevant Debian team. My dream is to raise such relevant teams inside Debian ... the Blends. Finally the last BoF of this series was the Debian Games meeting. As always I presented the team metrics graphs and the Debian Games team members who attended the BoF were quite interested. So it seems to be some unknown fact that team metrics are done for several teams in side Debian and so I repeat the link to it for those who are not yet aware of it. As a result of the BoF Debian Games team members agreed to put some more effort into maintaining their Blends tasks. Moreover Miriam Ruiz wants to put some effort into reviving Debian Jr. Regarding Debian Jr. there was an interesting talk about DouDouLinux - in case you might want to watch the video I'd recommend skipping the first 30min and rather watch the nice live demo. There was also an ad hoc BoF about Debian Jr scheduled to bring together all people interested into this cute project and Per Anderson volunteered to take over the lead. I have given a summary about this specific BoF at the Debian Jr list. For some other talks that I'd regard as remarkable for some reasons: I'd regard the talk "Debian-LAN" by Andreas Mundt as some hidden pearl because it did not got a lot of attention but after having seen the video I was quite impressed - specifically because it is also relevant for the Blends topic. Memories I also liked "Paths into Debian" by Moray Allan (and I was only able to enjoy the latter talks thanks to the great work of the video team!) because it also scratched the same topic I was concerned about in my mentoring talk. Related to this was in my opinion also "Women in Debian 2013" were we tried to find out reasons for the lack of woman compared to other projects and how to overcome this issue. Geert hovering  over the grass Besides the talks I will probably never forget two specific moments that make DebConf so special. One of these moments is recorded on an image that clearly needs no words - just see Geert hovering over the grass. Another strong moment in my personal record was in the DebConf Newbies BoF "First time at DebConf" that unfortunately was not recorded but at least for this statement it would have been very great if we would have some reference better than personal memory. Aarsh Shah a GSoC student from India suddenly raised up and said: "Four months ago I was not even aware that Free Software exists. Now I'm here with so many people who are totally equal. If I will tell my mother at home that I was standing in the same queue where the Debian Project Leader was queuing up for food she will never believe me." He was totally excited about things we are regarding as normal. IMHO we should memorise moments like this that might be part of the key to success in cultures, where Debian is widely unknown and very rarely in use. Amongst these not scheduled great moments the scheduled day trip was also a great thing. I had a really hard time to decide what tour I might join but ended up in the "long distance walking (or should I say running) group". Inspired by the "running Bubulle" who was flashing between the walking groups we went uphill with 5.4km/h which was a nice exercise. Our destination the large cliff was an exciting landscape and I guess we all enjoyed the dinner organised by the "Trout cabal". ;-) say goodby to  friends So I had a hard time to leave Le Camp and tried hard to make sure my memories will remain as long as possible. Keeping some signs attached to my bicycle, conserving the "Scenic Hacklab" sign for my private "scenic hacklab @ home" was one part. I also have cut some branches of the Buxus sempervirens in Le Camp and have put them in my garden at home (where I create some hedgerow from places where I spent some great time). These will probably build a great part of the hedgerow ... Thanks for reading this longish report. Looking forward to see you all in Germany 2015 (or earlier) Andreas. Scenic Hacklab  @ home

23 February 2013

Dirk Eddelbuettel: Two papers about RcppEigen and RcppArmadillo published

Two papers got published recently. The first one is Bates and Eddelbuettel (2013). It is titled Fast and Elegant Numerical Linear Algebra Using the RcppEigen Package, and provides a pretty thorough introduction to our RcppEigen package which uses Rcpp to provide access to the Eigen C++ template library from GNU R. The paper is out as Volume 50, Issue 5 at the (all electronic, open, and generally awesome) Journal of Statistical Software. A bibtex entry is available. The second paper is Eddelbuettel and Sanderson (2013). This one is titled RcppArmadillo: Accelerating R with high-performance C++ linear algebra and introduces the RcppArmadillo package which brings Conrad Sanderson's Armadillo C++ template library to GNU R by deploying Rcpp. The paper is currently "in press" at Computational Statistics & Data Analysis but the DOI 10.1016/j.csda.2013.02.005 will remain once a volume and issue is assigned by CSDA. Preprints of both papers are available via my papers page, and as vignettes in the corresponding packages. The upcoming Rcpp class in New York will feature Rcpp, RcppArmadillo and RcppEigen. Space is still available.

24 November 2012

Russ Allbery: Review: Dune

Review: Dune, by Frank Herbert
Series: Dune #1
Publisher: Ace
Copyright: 1965
Printing: September 1990
ISBN: 0-441-17271-7
Format: Mass market
Pages: 537
The Atreides family, distant cousins to the imperial family, have ruled the planet of Caladan for twenty generations. Caladan is a wet farming world, comfortable and pleasant, but not horribly important. But House Atreides is feuding with House Harkonnen, and, at the start of Dune, that feud maneuvers Duke Leto into giving up his holdings and moving his family to take possession of Arrakis. Arrakis is a desert planet, previously controlled by Baron Harkonnen. It is unrelentingly hostile, home to smugglers and dangerous local desert dwellers called Fremen. But it's also one of the most important planets in the galaxy, since it's the sole origin of the chemical called melange, or spice. Spice permits a limited form of prescience, which allows the navigators of the Spacing Guild to successfully steer ships across the interstellar void. Arrakis's production of spice is what makes interstellar travel, and therefore all of interstellar civilization, possible. Dune is the story of Paul Atreides, son and heir to Duke Leto Atreides. His mother, Lady Jessica, is one of the Bene Gesserit, a secretive order of women devoted to mental and physical discipline and to the long-term genetic improvement of mankind. He is not supposed to exist; Lady Jessica was supposed to only bear a daughter of Leto. But he may be something special, the long-sought (but also dangerous) Kwisatz Haderach who can unite male and female Bene Gesserit powers. The Bene Gesserit take great interest in him from the start of Dune. More surprisingly, so do the Fremen of Arrakis; from the moment he arrives there, he seems to be fulfilling prophecies of theirs that are partly, but not entirely, ones planted by the Bene Gesserit long ago. The feud with the Harkonnens, the unstable place of Arrakis in galactic politics, the dreams of the Fremen and the Imperial ecologist on Arrakis of terraforming, Bene Gesserit plans, Paul's abilities, and the legends of the Fremen all combine in a complex mix of politics, battle, and clashes of culture. Dune is an acknowledged SF masterpiece, one of the best-known classics of the genre. It's usually found in short lists of the best SF novels ever written. It spawned five sequels by Frank Herbert (about which more in a moment), as well as numerous additional sequels and prequels by Kevin J. Anderson and Brian Herbert. It's been adopted for the screen twice, not to mention board games, video games, and numerous other projects. This is my second reading, the first in about twenty years, but the story was still immediately familiar from having seen films and having discussed and read about the universe. This is not science fiction in any strict sense. Dune is science fiction in the same way that Star Wars is: a futuristic gloss on top of power structures inspired by feudalism, heavily mixed with mysticism, mental powers, magic, and implausible but convenient science that creates the story effects the author wants. Both Bene Gesserit powers in general and Paul's abilities in particular are effectively magic. There is some hand-waving explanation of their ability to verbally control other people as taking advantage of specific pitches and intonations that people are vulnerable to, but it's effectively spell-casting (and is a direct inspiration for Jedi mind tricks). All of the mysticism (and there's quite a lot of it in Dune, including race memory, precognition, and even molecular transformation) resembles the Force from Star Wars more than anything scientific. Dune is epic fantasy told on a science fiction stage, complete with a young protagonist coming into his powers and dangerous and sometimes hostile mentors. What Dune gets right, and what has put it so high in the pantheon of great science fiction, is the world building. Herbert sets the story tens of thousands of years into the future of humanity and then effectively projects the feeling of deep history over everything in the novel. This is the kind of book that has appendices with more background information; more to the point, it's the kind of book where you may actually read them out of curiosity. Mankind has a vast interstellar empire (Herbert's universe, like Asimov's Foundation universe, admits no aliens) governed by a system akin to the early British monarchy. An emperor rules in balance with the Great Houses, who meet in a sort of parliament. But against both is a third force: the Spacing Guild, who maintains a monopoly over all interstellar travel. (And the Bene Gesserit form an underground, secretive fourth power base.) Herbert plays with vast swaths of time and great forces of history as well as very good epic fantasy and better than nearly all SF I've read. The detailed world-building is equally good. Nearly all of Dune takes place on the desert planet of Arrakis, which has a lovingly-described ecology and local culture built entirely around scarcity of water. (The details of that ecology are much of the plot and mystery of the book, so I won't spoil them further.) While I doubt the precise details hold up to close scientific scrutiny, this is an obvious precursor to the great ecological stories of later SF, such as Kim Stanley Robinson's Mars trilogy. The details all feel right and hang together in satisfying ways, while also generating the great Sand Worms of Arrakis, a key ingredient in several of the best set pieces in the history of SF. This is the sort of book where the fascinating details and discoveries about the world do as much to keep one turning the pages as the plot, although the plot is also satisfyingly twisty and tense. Unfortunately, Dune doesn't get everything right. The amount of mysticism involved is a bit much, and at times the drug-trip mystical experiences of viewpoint characters turn into excessively purple prose and nearly incomprehensible descriptions. Those mystical experiences also involve race and genetic memory, a concept that's just scientific enough to be unbelievable. A few of the other scentific cheats are also rather blantant; for example, Herbert constructs an elaborate, artificial technology of shielding that seems designed primarily as an excuse to add sword combat to a futuristic story, and I have always struggled to suspend disbelief about the way lasers and shields interact in Dune. The Spacer Guild's monopoly on interstellar travel can be explained; their monopoly on local orbital space, or even the high stratosphere, both vital to allow certain things on Arrakis to remain secret, are much more dubious. Herbert mostly doesn't try to explain these things, and as with Star Wars the less explained the cheats are, the better they work as part of the story. But the technological background doesn't hold up against much examination. Worse, for me, is the general quality of the writing. Herbert does some things very well, such as world-building, and avoids awkward infodumps. Characterization and pacing are both fairly solid; he does a good job with Paul and Jessica in particular, and I've always liked the Fremen. But he wants to put the reader in everyone's head, frequently by giving character thoughts as italicized dialogue, and to enable that he uses a perspective that I always find distracting. Most fiction is written in tight third person. This means that the viewpoint character for any given section of the book is referred to in the third person, like all the other characters, but the reader has special access to their thoughts and emotions. We get to know what they're really thinking and feeling, not just the impressions they give to others, while the non-viewpoint characters are shown only from external appearances and the thoughts of the viewpoint character. Some books hold to the same viewpoint character throughout, but more commonly books move between viewpoint characters at scene breaks to provide more angles on the book's events. First person, in which the story is told by a specific character as if they were telling a story or writing it down, is the most common alternative. Third person objective, in which we don't get any special insight into the internal thoughts of any of the characters, is less common but still unsurprising. Dune does not use any of those perspectives. Instead, Dune uses wandering third-person omniscient, in which we get the inner thoughts and emotions of a character in a scene and then a few lines later the inner thoughts and emotions of a different character. This is the sort of thing that may or may not bug you depending on how much you've read, how deep the expectations of perspective are ingrained, and how much you notice perspective. It drives me nuts. I subconsciously align with the viewpoint character of a section, and pay attention to the ways that authors indicate which character will be the viewpoint character at the start of a scene. Herbert's constant flitting from character to character makes me dizzy. We get the verbatim thoughts of everyone almost indiscriminately, making me feel like I'm randomly hopscotching through the scene. For me, this does two things: it hurts my ability to get engrossed in the story, since I'm constantly thrown out of my normal reading mode when the viewpoint unexpectedly shifts, and it makes the writing feel repetitive. One keeps hearing about the same thing from multiple perspectives, and at times the story bogs down in everyone's internal dialogues rather than showing character reactions and letting the reader draw their own conclusions. I think it tries for a cinematic perspective, but ends up making the story feel muddled. The other flaw, which I didn't notice originally but which leaped out at me during this re-read, is that Herbert's world-building uses quite a few stereotypes. The most notorious, and most widely discussed, is of course the Fremen. Herbert draws heavily on Arab and Islamic culture even beyond the obvious similarities of people living in a harsh, arid climate. He borrows some rather loaded terms and cultural markers, such as jihad, to construct a culture of potential religious fanatics. This is not all bad; the Fremen are clearly portrayed as the good guys, which is a refreshing change from more typical current portrayals of Islam. But it becomes clear that they have aligned their entire culture around influences from outside, and the whole plot of Dune can be fairly characterized as an instance of "what these people need is a white man." Paul (and Kynes before him) joins their culture as well, but Paul becomes a better native than the natives, while simultaneously bringing his outside perspective. It's the sort of plot that is more widely noticed today than it would have been in 1965. Another major example of this, and one that I found more blatant, is that Herbert turns the Harkonnen into hissable, one-sided villains and uses some nasty stereotypes to do it. The insane torturer is consistently and repeatedly described as effeminate, fat is used as a marker of moral inferiority and evil, and the primary villain is homosexual and prefers drugged young male slaves. Here too, this sort of characterization short-cut was more common in 1965, but it's not appealing and makes the (already rather camp) scenes set among the Harkonnen even less enjoyable. Less clear-cut is the way women are handled throughout Dune. I do have to give Herbert some credit, particularly for the era in which he was writing. There are powerful female characters in Dune, including both Jessica and Alia, who have their own independent power and successfully pursue their own agendas throughout. The effectively all-female Bene Gesserit is a major political power in the story and is treated by the other players with respect as well as fear. But it's hard not to also notice the general position of women as subservient to men, not only in the general culture of the Great Houses but also in the more positively-portrayed Fremen culture. Indeed, the subservience of women is even worse in Fremen culture, where they're treated like property and where being killed by a woman is a sign of shame. Again, Herbert deserves some credit for doing better than a lot of 1960s fiction, but the sexism fairy has still been at work here. None of these flaws change the fact that Dune is a masterpiece. Herbert brings together history, world building, ecology, politics, and a compelling coming-of-age story about a messiah figure into a fast-paced, sweeping epic with a thoroughly satisfying conclusion. I think they do make it a flawed masterpiece, but it's still one of those SF novels that everyone should read at least once. Sadly, it's also a masterpiece that I think has suffered from its own success in the form of sequels, prequels, and a ton of supporting material. This is one of the problems that truly excellent world building can lead to. Human history is fractal: any specific detail can be examined in more depth and will usually lead (provided that information is available at all) to even more fascinating detail. The best world building conveys that impression of depth. That's what Herbert achieves here with hints, notes, and asides: the sense that galactic history is a vast ediface with the same fractal complexity as real human history. It makes for a compelling background, but it also inspires people to dig into that background and flesh out all of the details the way that we do with human history. But this doesn't actually work; invented history created by one person simply cannot be fractal in the same way. Human history is endlessly complex because it was generated by the complex interactions of many people. Invented history is an illusion that hints at complexity by building the same surface, but one mind, or even a small number of minds, cannot generate the same depth. The result is that if one digs too deep, one removes that convincing surface and ends up with a mundane, simplistic, and unsatisfyingly fake set of events. I think that's what's happened with all of the supporting material that's been written around Dune since its original publication. Dune is of a piece, a single story that's deeply enjoyable on its own terms and leaves the reader with a satisfying impression of complexity. The systemic excavation of that complexity lessens it and reveals too much of the illusion. Yes, I want to know more about the Butlerian Jihad, but that's the point: the wanting is the sign of succesful crafting of imagined history. Reading the definitive account is more likely to leave me unsatisfied than to lead to the recursive curiosity that human history can create. The sequels to Dune written by Herbert himself are, for me, another matter. Some reviewers level the same criticism at them: that Herbert dives too far into background best left unexplored. But they have the advantage of moving forward, telling more of the story set off by Paul, and the end of Dune is a clear setup for a sequel. One of Paul's goals throughout most of the book has been left unaccomplished. I don't think Herbert dove too deep into his creation; rather, my problem with his sequels (all of which I've read, although it's been some years now) is that he took the story in a direction that I actively disliked and found painful to read. Regardless, the general consensus is that the sequels aren't as good as the original, and while Dune doesn't fully resolve its story, it's complete enough that it's possible to stop here. Stopping is the general recommendation, although I still may re-read and review the sequels at some point. Followed by Dune Messiah. Rating: 8 out of 10

3 July 2012

Dirk Eddelbuettel: RcppArmadillo 0.3.2.3

Conrad releaser version 3.2.3 of Armadillo a few days ago, and the corresponding RcppArmadillo package 0.3.2.3 is now CRAN. (For these keeping score 3.2.1 never was a full release, and 3.2.2 containing fixes for a build issue that did not affect the R package build so we skipped it.) The short NEWS entry follows below. This version fixes some issues related to g++ 4.7; however the current Debian testing version of g++-4.7.1 required that I rolled back three header files to the version from the previous release. Conrad has been in contact with the gcc upstream maintainers and a fix may appear in time for g++-4.7.2. This release also contains a new introductory pdf vignette based on a paper Conrad and I just submitted. It introduces Armadillo to R programmers and demonstrates with a simple Kalman filtering example how the code can be written in the same concise matrix-oriented style, yet runs orders of magnitude faster.
0.3.2.3  2012-07-01
    o   Upgraded to Armadillo release 3.2.3 
          * minor correction for declaration of fixed size vectors and
            matrices
    o   Reverted three header files  Mat,Row,Col _bones.hpp back to previous
        release due to compilation failures under g++-4.7
    o   Added new vignette 'RcppArmadillo-intro' based on a just-submitted
        introductory paper (by Eddelbuettel and Sanderson) about RcppArmadillo 
    o   Change from release 3.2.2 which we skipped as it did not really affect
        builds under R:
          * minor fix for compiling without debugging enabled (aka release
            mode)
          * better detection of ATLAS during installation on Fedora and Red
            Hat systems
    o   Small enhancement to fastLm 
Courtesy of CRANberries, there is also a diffstat report for 0.3.2.3 relative to 0.3.2.0 As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

5 May 2012

Dirk Eddelbuettel: RcppArmadillo 0.3.0.3

Two days ago, Conrad Sanderson released another bug-fix version 3.0.3 for the 3.0.0 branch of his excellent Armadillo C++ template library for linear algebra. The new RcppArmadillo release 0.3.0.3 which contains it appeared on CRAN yesterday. Beside Conrad's bugfixes we also added an example script for fastLm and faster linear model fits. The short NEWS entry follows below.
0.3.0.3 2012-05-03
    o   Upgraded to Armadillo release 3.0.3
          * fixes for inplace transpose of complex number matrices
          * fixes for complex number version of svd_econ()
          * fixes for potential aliasing issues with submatrix views
    o   New example script fastLm 
Courtesy of CRANberries, there is also a diffstat report for 0.3.0.3 relative to 0.3.0.2 As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

20 April 2012

Dirk Eddelbuettel: RcppArmadillo 0.3.0.2 released and on CRAN

Earlier today, Conrad Sanderson released another bug-fix version 3.0.2 for the still fairly recent 3.0.0 version of his excellent Armadillo C++ template library for linear algebra. The new RcppArmadillo release 0.3.0.2 also appeared on CRAN this morning. Beside Conrad's bugfix in Armadillo itself, he also convinced us to unroll a change imposed by R: NDEBUG is now being defined unconditionally when compiling R packages. In Armadillo's case, this suppresses a number of useful things including bounds-checking. So we now undefine this symbol in the initial RcppArmadillo headers. Users can still set it manually, and/or define ARMA_NO_DEBUG. The short NEWS entry follows below.
0.3.0.2 2012-04-19
    o   Upgraded to Armadillo release 3.0.2
          * fixes for handling diagonal matrices
    o   Undefine NDEBUG if it has been set (as R does) as this prevents a
        number of useful debugging checks. Users can still define it or
        define ARMA_NO_DEBUG if they want a 'non-development' build
Courtesy of CRANberries, there is also a diffstat report for 0.3.0.2 relative to 0.3.0.1 As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

12 April 2012

Dirk Eddelbuettel: RcppArmadillo 0.3.0.1 released and on CRAN

Conrad Sanderson released a bug-fix version 3.0.1 following up on the very recent 3.0.0 version of his excellent Armadillo C++ template library for linear algebra. I made a new RcppArmadillo release 0.3.0.1 which just appeared on CRAN. The short NEWS entry follows below.
0.3.0.1 2012-04-12
    o   Upgraded to Armadillo release 3.0.1
          * fixes for compilation errors
          * fixes for potential aliasing issues
Courtesy of CRANberries, there is also a diffstat report for 0.3.0.1 relative to 0.3.0 As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

11 April 2012

Dirk Eddelbuettel: RcppArmadillo 0.3.0 is now on CRAN

Conrad Sanderson has released a new major version 3.0.0 of his excellent Armadillo C++ template library for linear algebra. A corresponding new release 0.3.0 of RcppArmadillo is now on CRAN. This follows four pre-releases of Armadillo which we packaged as before, but for reasons detailed in another post did not send to CRAN. All that said, here are the NEWS entries for all release since the last non-test release:
0.3.0   2012-04-10
    o   Upgraded to Armadillo release 3.0.0 "Antarctic Chilli Ranch"
          * added non-contiguous submatrix views
          * added shorthand for inverse: .i()
          * added hist() and histc()
          * faster repmat()
          * faster handling of submatrix views with one row or column 
          * faster generation of random numbers
          * faster element access in fixed size matrices
          * better detection of vector expressions by sum(), cumsum(),
            prod(), min(), max(), mean(), median(), stddev(), var() 
          * expressions X=A.i()*B and X=inv(A)*B are automatically converted
            to X=solve(A,B) 
0.2.40  2012-04-04
    o   Upgraded to Armadillo release 2.99.4 "Antarctic Chilli Ranch (Beta 4)"
          * fixes for handling expressions with fixed size matrices
0.2.39  2012-04-02
    o   Upgraded to Armadillo release 2.99.3 "Antarctic Chilli Ranch (Beta 3)"
          * faster repmat()
          * workarounds for braindead compilers (eg. Visual Studio)
0.2.38  2012-03-28
    o   Upgraded to Armadillo release 2.99.2 "Antarctic Chilli Ranch (Beta 2)"
          * added .i()
          * much faster handling of .col() and .row()
          * expressions X=A.i()*B and X=inv(A)*B are automatically converted
            to X=solve(A,B) 
0.2.37  2012-03-19
    o   Upgraded to Armadillo release 2.99.1 "Antarctic Chilli Ranch (Beta 1)"
          * added non-contiguous submatrix views
          * added hist() and histc()
          * faster handling of submatrix views
          * faster generation of random numbers
          * faster element access in fixed size matrices
          * better detection of vector expressions by sum(), cumsum(),
            prod(), min(), max(), mean(), median(), stddev(), var() 
Courtesy of CRANberries, there is also a diffstat report for 0.3.0 relative to 0.2.36 As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

17 February 2012

Dirk Eddelbuettel: RcppArmadillo 0.2.35

Now that Rcpp 0.9.10 is released and on CRAN, other packages can take advantage of a small change needed to make use of the quasi-output stream Rcpp::Rcout. So the new release 0.2.35 of RcppArmadillo does just that---and input/output from Armadillo, the wonderful linear algebra / math library for C++ written chiefly by Conrad Sanderson now comes through in a coordinated buffered fashion. As as std::cout is no longer used by default, the R CMD check no longer flags this. There are two more bugfixes in the releases for issues noticed by attentive RcppArmadillo users. Teo Guo Ci spotted a missing semicolon when C++0x code was activated, and Gershon Bialer suggested a fix for an issue plagueing users of the jurassic g++ 4.2.1 compiler which Apple forces on users of OS X. We thank Martin Renner for testing these fixes. The NEWS entry below summarises the changes.
0.2.35  2012-02-17
    o   Upgraded to Armadillo release 2.4.3
          * Support for ARMA_DEFAULT_OSTREAM using Rcpp::Rcout added
    o   Minor bug fix release improving corner cases affecting builds:
          * Missing semicolon added in Mat_meat (when in C++0x mode), with
            thanks to Teo Guo Ci 
          * Armadillo version vars now instantiated in RcppArmadillo.cpp
            which helps older g++ versions, with thanks to Gershon Bialer
          * Thanks also to Martin Renner for testing these changes
          * Unit tests output fallback directory changed per Brian Ripley's
            request to not ever use /tmp
          * Minor update to version numbers in RcppArmadillo-package.Rd
Courtesy of CRANberries, there is also a diffstat reports for 0.2.35 relative to 0.2.34 As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

14 December 2011

Dirk Eddelbuettel: RcppArmadillo 0.2.34

And another quick bugfix release by Conrad Sanderson made it version 2.4.2 bug of Armadillo. And this is in RcppArmadillo release 0.2.34 which got to CRAN this morning The NEWS entry below summarises the changes.
0.2.34	2011-12-12
    o   Upgraded to Armadillo release 2.4.2
          * clarified documentation for .reshape()
          * fix for handling of empty matrices by .resize()
Courtesy of CRANberries, there is also a diffstat reports for 0.2.34 relative to 0.2.33 As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

8 December 2011

Dirk Eddelbuettel: RcppArmadillo 0.2.33

Conrad Sanderson made a first 2.4.1 bug fix release of Armadillo. We now have RcppArmadillo release 0.2.33 on CRAN which contains it. earlier today. The NEWS entry below summarises the changes.
0.2.33  2011-12-07
    o   Upgraded to Armadillo release 2.4.1
          * added .resize()
          * fix for vector initialisation
Courtesy of CRANberries, there is also a diffstat reports for 0.2.33 relative to 0.2.32 As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

6 December 2011

Dirk Eddelbuettel: RcppArmadillo 0.2.32

Conrad Sanderson made the official 2.4.0 release of Armadillo last evening. This was wrapped into RcppArmadillo release 0.2.32 which arrived on CRAN earlier today. The NEWS entries summarising the changes since the 2.2.* series, we already saw most of this with the two prerelease 0.2.30 and 0.2.31:
0.2.32  2011-12-04
    o   Upgraded to Armadillo test release 2.4.0 "Loco Lounge Lizard"
    o   Minimal changes relative to 0.2.31 based on 2.3.92, next section is
        relative to the previous stable release series 2.2.* of Armadillo
          * added shorter forms of transposes: .t() and .st()
          * added optional use of 64 bit indices, allowing matrices to have
            more than 4 billion elements 
          * added experimental support for C++11 initialiser lists
          * faster pinv()
          * faster inplace transpose
          * faster handling of expressions with diagonal views
          * fixes for handling expressions with aliasing and submatrices
          * fixes for linking on Ubuntu and Debian systems
          * fixes for inconsistencies in interactions between matrices and
            cubes 
          * refactored code to eliminate warnings when using the Clang C++
            compiler 
          * .print_trans() and .raw_print_trans() are deprecated
Courtesy of CRANberries, there is also a diffstat reports for 0.2.32 relative to 0.2.31 As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

30 November 2011

Dirk Eddelbuettel: RcppArmadillo 0.2.31

Conrad Sanderson just released the second pre-release 2.3.92 of what will be Armadillo 2.4.*. This is now in RcppArmadillo release 0.2.31 which is already on CRAN as of this morning. The NEWS entries summarising the changes for both are below:
0.2.31  2011-11-28
    o   Upgraded to Armadillo test release 2.3.92 "Loco Lounge Lizard (Beta 2)"
          * fixes for linking on Ubuntu and Debian systems
          * fixes for inconsistencies in interactions between matrices and
            cubes
Courtesy of CRANberries, there is also a diffstat reports for 0.2.31 relative to 0.2.30 As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

Next.

Previous.